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CN . Abstract. The Restricted Isometry Property (RIP) introduced 

I by Candes and Tao is a fundamental property in compressed sens- 

ed . ing theory. It says that if a samphng matrix satisfies the RIP of 

certain order proportional to the sparsity of the signal, then the 
. original signal can be reconstructed even if the sampling matrix 

provides a sample vector which is much smaller in size than the 
original signal. This short note addresses the problem of how a 
linear transformation will affect the RIP. This problem arises from 
, the consideration of extending the sensing matrix and the use of 

^ ' compressed sensing in different bases. As an application, the re- 

sult is applied to the redundant dictionary setting in compressed 
sensing. 
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In Compressed Sensing (CS), one considers the problem of recovering 
a vector (discrete signal) x G from its linear measurements y of the 
form 

(1.1) t/i =<x,ipi>, I <i < n, 

with n « N. If a; is sparse, CS theory says that one can actually 
recover x from y which is much smaller in size than x by solving a 
convex program with a suitably chosen set of sampling row vectors 
{(pi\l < i < n} [1][2][3]. The linear system (11. II) can be written in the 
form of matrix multiplication 

(1.2) y = <Dx, 

where $ is an n x matrix formed by the row vectors (pi called the 
sampling matrix. One of the Conditions that ensures the performance 
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of the samphng matrix $ is the RIP. A matrix $ G M"^^ is said to 
satisfy the RIP of order A; G N and isometry constant 6k G (0, 1) if 

(1.3) (1 - 6k)\\z\\l < \\<i>Tz\\l < (1 + 6k)\\z\\l, \fz G Rl^l, 

where T C {1, 2, ... , A^} satisfying |T| < k, and $r denotes the ma- 
trix obtained by retaining only the columns of $ corresponding to the 
entries of T. Condition (11. 3p is equivalent to the condition that all the 
matrices have their eigenvalues in [1 — 5fc, 1 + 6k]- For any ma- 

trix X G W^'"^ and any G N, we denote the corresponding isometry 
constant of X by 6k{X). If there is no confusion, we will just write 6k- 
In particular, we always use 6k for the matrix $. 

A theorem due to Candes, Romberg, and Tao [1] says that if $ 
satisfies the RIP of order 3k, then the solution x of the following convex 
minimization problem 

(1.4) min||a;||i subject to = y, 
satisfies (see also ^) 

(1.5) \\x-xh<^''''^^^ 



Vk ' 

where ak{x) is the ii error of the best fc-term approximation, and C2 
is a constant depending only on G (0, 1)0 

A condition that ensures a random matrix satisfies the RIP with 
high probability is given by the concentration of measure inequality. 
An n X N random matrix $ is said to satisfy the concentration of 
measure inequality if for any x G M^, 

(1.6) i'dll^a;!!^ - ||x||^| > £||a;||^) < 2e-""o("\ 

where e G (0, 1), and Co{e) is a constant depending only on e. 

The random matrices $ = (rj^) generated by the following proba- 
bility distributions are known to satisfy the concentration of measure 
inequality with Co{e) = — e^/Q [5J: 

r 



~ AT 0, 



— with probability 1/2 
with probability 1/2 



n 



should be noted that the RIP is only a sufficient condition for reconstruction. 
If $ satisfying the RIP, cA may not satisfy the RIP for c 7^ 0. However, it is clear 
that both A and cA lead to similar sparse recovery using ii program. However, 
this issue is beyond the current scope [5]. 
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According to Theorem 5.2 in [5ji . for given integers n and N , and 
< 5 < 1, if the probabihty distribution generating the nx N matrices 
$ satisfies the concentration inequahty (11. 6p . then there exist constants 
Ci, C2 > depending only on S such that the RIP holds for $ with the 
prescribed 6 and any 

(1.8) A; < cin/ log(A^/A;) 

with probability > 1 — e~^'^^". Furthermore, this RIP for $ is universal 
in sense that it holds with respect to any orthogonal basis used in the 
measurement. 

There are also deterministic constructions of matrices satisfying the 

RIP [Z][H][9][in]. 

For application purposes, one often needs to analyze the RIP con- 
stants of the products of a matrix $ with known RIP constant 6 and 
other matrices. For example, when one considers different bases or 
redundant dictionaries under which the signals of interest are sparse, 
matrices of the form $5 needs to be analyzed [2] [11] , where B is given 
by the basis or the dictionary. For another example, if the size of $ is 
n X N with n < N, one would like to extend $ to A^B of size m x q 
with m<n<A^<gif possible, since that gives a further reduction on 
the number of measurements one needs to collect: for $, the number 
of measurements is n; while for A^B, the number of measurements is 
m. 

These situations can be formulated under a more general framework 
by asking the following question: If a matrix $ of size nxN satisfies the 
RIP with a given isometry constant < 6 < 1 (with certain probability 
if $ is random), and A, B are given matrices of sizes mx n and N x q 
respectively, then what is the isometry constant of the matrix A^^Bl 

In section 2, we first show that if all A, and B are random and 
satisfy the concentration of measure inequality, then A(^B satisfies the 
concentration of measure inequality, therefore it has RIP. Then we 
observe that if deterministic matrix is involved, the problem is more 
complicated, but it can still be analyzed by using the SVDs of A and 
B. It is not possible to multiply by a deterministic A from the left 
to achieve more reduction on the number of measurements without 
further assumption. Our result shows that it is possible to extend the 
matrix $ by multiplying a deterministic B from the right to extend $ 
if $ is random, though the isometry constant will be changed. This 



In the proof given in [5], the constant ci was first chosen such that a := 
co(e)5/2 — ci[l + (1 + log -y)/ log Q, then the constant C2 was chosen such that 
< C2 < a. Thus the constants depend also on e. 
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result can be applied to redundant dictionary setting to give a different 
approach for using CS with redundant dictionaries. 

2. Main result 

We first consider the random case. Let $ be an n x matrix sat- 
isfying the concentration inequality (11.61) with constant e, and let A 
(respectively B) be a random matrix size m x n (respectively N x q) 
satisfying the concentration inequality (11. 6p with ei (respectively, €2). 
Then we have: 

Theorem 2.1. Assume that alle,ei,e2 < 1/3. The matrix A^ satisfies 
the concentration inequality 

P{\\\A<^>x\\l - \\x\\l\ > esWxWl) < 2e-™'=o, 

where £3 = e + ei{l + e) , and c'q is a constant that depends only on 
Co(e) and CQ{ei) (as defined in U.6\) ). The same statement holds for 
$i? with 63 = e + 62{1 + e) and m replaced by n. 

Proof. We give the proof for the case of left multiplication by A, the 
proof for the case of right multiplication by B is similar. By assump- 
tion, with probability > 1 — 2e~™''^°^^^\ the matrix A satisfies 

(1 - e,)\\y\\l < \\Ay\\l < (1 + e,)\\y\g for any y G M". 

Replacing y by $x {x G M^), we have 

(2.1) {1 - e^)\\<l>x\\l < \\A<l>x\\l < {1 + e,)\\<!>x\\l 

Again by assumption, with probability > 1 — 2e~"^°^^\ the matrix $ 
satisfies 

(2.2) (1 - e)\\x\\l < \\<!>x\\l < (1 + e)\\x\\l, for any x G R^. 

Now the statement follows by combining (12.11) and (12. 2p . □ 

Remark. If m < n, the constant Cq in Theorem 2.1 can be roughly 
estimated by the inequality c'q < Cq^e') — log2/m, where Cq^e') = 
min{co{Ei),Co{E)}. This is obtained from 

]_ _ (^]_ _ 2e~'"'^"''^^'')(l — 2e~"'^''''^-*) < 2e~'"*-'^°*-^'-*~'°^^/'^^ 

More precise estimation can be carried out, but we are not concerning 
this point here. 

Now we consider the cases when deterministic matrices are involved. 
We observe that it is not possible to multiply a deterministic matrix A 
from the left to extend the sensing matrix to achieve further reduction 
in sampling without other assumptions. To see this, we consider the 
SVD of A. 
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For any positive integer d, let 0{d) be the set of d x d orthogonal 
matrices. There exists U G 0(n) such that 



(2.3) 



A' A = W 



V 



0"2 



\ 



u, 



where o"i > o"2 > ■ ■ ■ > cr„ > 0. Since for any T C {1,2,..., N}, 

{A^)t = A^T, we have 

(2.4) (A$)^(A$)t = $^^*A$r 



0-2 



a. 



n J 



( ^1 



V 



0-2 



(f/$)^ 



If m < n, then o"m+i 



cr„ = 0, and hence 



0-2 



\ 



On ) 



(t/$) 



T 







V 

for a suitable block matrix A\. From the last matrix one can see im- 
mediately that RIP fails. 

If m > n, then we can change $ by multiplying A from the left if A 
has full column rank. Since under this assumption, all (Xj > 0. Note 
that [/$ has the same RIP as $, so if 5/c is the RIP constant of $ 
corresponding to all T of size k < N, we can bound the RIP constant 
of y4$ by cr„(l — 5k) and cri(l + Sk)- In fact, for z E M^', if we let 
U^tz = y = {yi, • . • , ynY, then \\y\\2 = \\^tz\\2, and according to (12.41) 

n 

(2.5) aM\l < W^'^TzWl = Y^^iVi < ^M\l 



1=1 



Thus we have (use (11.31) ) 
(2.6) anil - 5k)\\z\\'] 



< \\A^Tz\\l<aiil + Sk 



Note that the above analysis works whether $ is random or determin- 
istic. 
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Next, we consider the product $5. In this case, we need to distin- 
guish between random matrix $ and deterministic matrix $. Assume 
that $ is a random matrix satisfying the concentration inequahty (11.61) 
and hence satisfying the RIP inequality (11.31) with probability > p. 
Note that the concentration inequality is invariant under the right mul- 
tiplication by an orthogonal matrix. That is, if U & 0{N), then $[/ 
also satisfies (II. 3p with probability > p. 

Let B he an N xq matrix. To make the argument clearer, we assume 
that T C {1, 2, . . . , g} with \T\ = k < N (note that this is sufficient for 
our purpose). We have U e 0{N) and V E 0{k) such that 



(2.7) 
where 

D = 

For X G 
(2.8) 
Let 



U 



D 




Nxk 



Xk J 



^Btx = $t/ 



D 




Ai > Aa > ■•■ > Afc > 0. 



D 




Vx. 



Nxk 



Vx G 



bN 



Then z is fc-sparse (the last N — k entries are always 0). Thus, since 
$?7 has the same RIP as $, we have 



(2.9) 



'1-6, 



\z\\l< 



I^Btx 



mzlli <(l + 6k 



with probability > p. 



Let y = {yi, ytf = Vx, then \\y\\2 = ||x||2, and 



x'V\D' 0) 



D 




A2 



v 



Vx 

\ 



i=l 
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Since 

A^ibii^<i:Ak^<A?ibiii 

i=l 

by (]2.9p . we have 

(2.10) Xlil - 6,)\\x\\l < mrxWl < A?(l + 6,)\\x\\l 

with probabihty > p. 

If $ is deterministic, then for arbitrary U G 0{N), $t/ may not 
satisfy the same RIP as $, and we do not have a good analysis of $i? 
for this case at the moment. Summarize our discussion, we have: 

Theorem 2.2. Notation as before. 

(1) If A is deterministic, then regardless whether $ is random or 
deterministic, ^4$ has RIP if and only if A has full column rank. If 
that is the case, the RIP constant of A^ can he obtained from li2.6\) . If 
$ is random, then the probability for A^ to satisfy RIP is the same as 
that of ^ (with possible different RIP constant). 

(2) If ^ is a random matrix satisfying the concentration inequality 
U.6]) (hence satisfying the RIP U.^) with probability at least p), and 
B is an N X q deterministic matrix such that Sk{B) G (0, j^), then 
with probability at least 

(2.11) l-(^)(l-p), 

the matrix satisfies the RIP with the same order as that of $ and 
a possible different RIP constant 6k{^B) determined by Ii2.10\) . 

3. Redundant Bases in Compressed Sensing 

In this section, we apply Theorem 2.2 to redundant bases setting in 
compressed sensing. From (11.81) . we see that for given and k, the 
random matrices of size nx N generated by the distributions described 
in (11.71) satisfy the RIP with high probability as long as 

n > Cklog{N/k) for some constant C. 

Therefore it is desirable to reduce the integer k, i.e. to increase the 
sparsity level of the signal, by considering redundant bases (or redun- 
dant dictionaries). Recall that if a set of vectors B spans a vector 
space V, then we call B a basis if B is linearly independent and call B 
a redundant basis otherwise. To apply compressed sensing to a signal 
y G that has a sparse representation x under a redundant basis 
B of size q > N, we need to consider how the combination of a good 
sensing matrix with a redundant basis affects the RIP. 
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Let B be the matrix corresponds to the redundant basis B. Then 
B is of size N x q and y = Bx with x G M'' sparse. This problem has 
been considered in [2] [11]. In particular, in [11], a detailed analysis of 
the situation was given. According to Theorem 2.2 in [TT], if $ satisfies 
the concentration inequality (11.61) with B 

(3.1) n > C5f[k{log{N/k) + loge(l + 12/5^)) + log2 + t], 

for some 6k G (0, 1) and t > 0, then with probability at least 1 — e~*, 
the restricted isometry constant of $-8 satisfies 

(3.2) 4($5) < 6k{B) + 6k{l + 6k{B)). 

We now apply Theorem 2.2 to obtain a similar result. 

Theorem 3.1. Notation as above. With the isometry constant satis- 
fying 

(3.3) Sk{<^B) < 5u{B) + 4($)(1 + k{B)) 

and the probability bound given by Ii2.11\) . the matrix ^B satisfies the 
RIP with the same order as that of ^. 

Proof. One just needs to note that the numbers and Ai which appear 
in (12^101) satisfy 

l-k{B)<\l<\\<l + 5k{B). 

□ 



For examples of redundant bases satisfying the condition in Theorem 
3.1, we refer the readers to |llj . 

4. Conclusion and Discussion 

We analyzed the problem of how the multiplication of a matrix to a 
good sensing matrix affects its RIP. This type of problems arise in CS 
when one wants to extend the sensing matrix by taking the product 
of the sensing matrix with another matrix. A particular interesting 
example is the apphcation of CS under the redundant bases setting. 
Our result in this short note provides some basic theory for further 
investigation on the RIP and its applications in CS under different set- 
tings. Future work includes constructing good redundant bases, which 
is related to constructing good deterministic sensing matrices, and an- 
alyzing their properties under CS. 

"'There should be a factor S (which is our k) for the term log(e(l + 12/(5)) in the 
bound for n given in [TT]. This affects some later estimates in [IT] . 
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